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MULTI-PASS INTERFERENCE REDUCTION 
IN A GSM COMMUNICATION SYSTEM 



FIELD OF THE INVENTION 

This invention relates generally to communication systems, and more 
particularly to reducing interference to a received signal in a communication system. 



BACKGROUND OF THE INVENTION 

Wireless communication systems have an ever-increasing use in transferring 
voice and data services. Such communication systems include the Global System for 
Mobile ("GSM") communication. However, stray signals, or signals intentionally 
10 introduced by frequency reuse methods, can interfere with the proper transmission 
and reception of voice and data signals and can lower capacity. As a result, a receiver 
must be capable of processing a signal with interference from at least one channel and 
extract the desired information sent to a user. It is well known that, for typical cell 
layouts, the major source of noise and interference experienced by GSM 
15 communication devices when the network is supporting a non-trivial number of users 
is due to co-channel or adjacent channel interference. Such noise sources arise from 
nearby devices transmitting on or near the same channel as the desired signal, or from 
adjacent channel interference, such as noise arising on the desired channel due to 
spectral leakage for example. Additionally, even in the case where no other signal 
20 interference is present, the received signal may consist of multiple copies of the 
transmitted data sequence for example due to multi-path channel conditions. This 
effect is sometimes referred to as self-interference. 

Traditionally, the effects of multi-path channels are compensated either 
through the use of equalization methods, such as the Maximum Likelihood Sequence 
25 Estimation ("MLSE") which is usually implemented using the Viterbi algorithm, or 
the Decision Feedback Equalizer, or through filtering techniques. In the filtering 
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approach, a desired signal symbol sequence can be estimated by adjusting the filter 
parameters. Classically, the filter parameters can be determined using the modulated 
symbol and training sequences contained within the desired signal using well known 
techniques, such as the Minimum Mean Square Error Block Linear Equalizer 

5 ("MMSE-BLE") for example, which operates on the complex values of the signal and 
generally can be implemented in the frequency and time domains. 

Traditionally, interference cancellation techniques have focused on adjacent 
channel suppression by using several filtering operations to suppress the frequencies 
of the received signal that are not also occupied by the desired signal. 

10 Correspondingly, co-channel interference techniques have been proposed, such as 
joint demodulation, which generally require joint channel estimation methods to 
provide a joint determination of the desired and co-channel interfering signal channel 
impulse responses. Given known training sequences, all the co-channel interferers 
can be estimated jointly. However, this joint demodulation requires a large amount of 

15 processing power, which constrains the number of equalization parameters that can be 
used efficiently. Moreover, classical joint demodulation only addresses one co- 
channel interferer, and does not address adjacent channel interference. 

Multiple antenna techniques have also been proposed but these can be 
complex in their terms of hardware implementation and therefore are mainly more 

20 suited to a base station application. Unfortunately, all of the above techniques are 
non-trivial in either implementation and/or complexity. 



-2- 



CS23196RL 



EXPRESS MAIL NO.: EL977215603US 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is an exemplary format of the GSM normal burst; 
FIG. 2 is an exemplary block diagram of a receiver with an alternate linear 
5 output equalizer filter, in accordance with the present invention; 

FIG. 3 is an exemplary block diagram of the alternate linear output equalizer 

filter of FIG. 2; 

FIG. 4 is an exemplary flowchart for describing a method for alternate linear 
output equalization, in accordance with the present invention; and 
10 FIG. 5 is an exemplary graphical representation of the performance 

improvement provided by the present invention. 



-3- 



CS23196RL 



EXPRESS MAIL NO.: EL977215603US 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention reduces co-channel and adjacent channel interference in 
a communication system using a linear finite-impulse-response ("FIR") filter with 
reduced computational complexity without sacrificing interference cancellation 
performance. The present invention uses a filter specially adapted to the Gaussian 
Minimum Shift Keying ("OMSK") modulation used for GSM signals. However, the 
filter is applicable to any modulation with a generally constrained modulation 
constellation, and can be implemented with low complexity. The basic unit of 
transmission for both circuit- and packet-switched GSM logical channels is the 
normal burst (other burst formats are defined in GSM, but are reserved for signalling, 
frequency correction or other purposes). The format of an exemplary normal burst, 
which is illustrated in FIG. I, comprises two tail bit fields ('T') 102 and 104 of length 
equal to 3 bits each, two encrypted data fields ("DATA") 106 and 108 of length-57 
bits each, two stealing bit fields ("S") 1 10 and 1 12 of length-1 bit each, the midamble 
or training sequence code ("TSC") 114 of length 26 bits, and the guard interval ('G') 
1 16 of nominal length 8.25 bits. The present invention uses quasi-linear processing, 
and further no additional hardware is required and processor requirements are reduced 
substantially compared to the traditional joint demodulation method. Specifically, a 
filter estimator is trained using symbols with a known specific phase in a received 
signal. More specifically, since the training symbols which comprise the midamble of 
length 26 bits are known to have a 90-degree phase difference and are therefore 
known to have only real or imaginary values, the training symbols provide a 
convenient means of estimating the optimal quasi-linear filter. Advantageously, by 
limiting the symbols to a constrained phase relationship, the complexity of 
calculations to be performed in order to linearly equalize the received signal is 
reduced. In the description below, any quantities {.f, (.)", (.)*^ represent the 
transposition, conjugate transposition, and inversion of matrices, respectively, and 
bold letters indicate vectors or matrices. One can define b(k) as a desired user's 
transmitted data sequence of symbols where 
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bik)i 



[{±l},^e{l, 3, 5,.- } 



In other words, each transmitted symbol Kk) in the GSM system, when OMSK 
modulation is used, consists of either an in-phase (I) or a quadrature (Q) component. 

5 A known string of training symbols is defined for a burst of data with OMSK 
modulation in the GSM system. This string corresponds to a set of twenty six 
symbols referred to as the midamble or training sequence, b(k): k e {62,63, -, 87}, that 
are known a priori, or known beforehand. The training sequence can be used to 
determine filter parameters to reduce interference, as will be described below. 

10 Due to co-channel, self and/or adjacent channel interference, the desired 

sequence, b{k\ can contain errors. The observation vector, y(k\ comprises the 
actually observed received signal represented by y(k). Then the midamble of the 
received signal, y(62,63,-,87), contains the actual received training sequence of the 
first hypothesized arriving ray of the received signal. An error is defined as the 

15 difference between the desired and received estimated sequence, b{k) andi?(^) , i.e. 
between what was originally sent and what is actually estimated by the receiver. A 
filter can be applied to the actually received signal, y[k), to minimize this error. For 
example, a complex linear estimate of b{k) may be defined as 

20 ^(^) = w"xy(/:) 

where w = [ w(0), w(l), • • • , w(L - l)f is a vector of complex-valued filter taps, 
y(Jt) = [y(it), y{k + !),•••, y{k + L - l)f is a sliding vector of received signal samples, 
and L is the number of taps of the filter. This complex linear tap estimate could then 
25 used with the observed signal to determine the error as follows 
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The traditional but less efficient approach for optimizing w is to minimize the 
complex sum- squared error, Err, using the known midamble training sequence bits. 

However, the present invention takes advantage of the fact that it is known 
a priori that b(k) is either purely real or imaginary, as a result of the GMSK 
5 modulation. Therefore, improved performance can be obtained by minimizing the 
sum-squared error using the modified error metric 



where b(k) is now always purely real or purely imaginary. Therefore, a novel aspect 
of the present invention, distinct from the traditional approach to equalization, is that 
15 the real/imaginary portion of the k-th equalizer output is taken as an estimate of the 
non-zero (real/imaginary) portion of the data symbol b(k). This approach finds 
particular advantage, in that interference products, which are typically subject to 
fading, are therefore subject to phase rotation distinguishable from the real-only or 

imaginary-only values of b{k). The use of the fixed-phase, real/imaginary portions of 

20 the midamble, provides a distinction that can then be used to eliminate the 

interference products. Moreover, the use of only real/imaginary data in minimizing 
the sum-squared error simplifies calculations. 

The linear estimator w can be optimized based on a least-squares estimate 
calculated using the following equation: 




where the linear estimate b(k) is defined as: 



10 



b(k) = 



rc^Kyv"ym.ke{h 3,5,..} 
imag(w''y(^)),^e{2,4, 6,---} 



25 



w={z''zr'z^t' 
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where w is a purely real vector of length 2L (for the particular case of symbol-spaced 
operation, and where the approach is readily extended to fractional filter tap-spacing) 
containing the L real values and the L imaginary values of the linear filter, t' is a 
vector containing the non-zero real and imaginary components of the training 
5 sequence of known symbols, and Z is the observation matrix of the received signal 
values. The matrix Z and the vector t' can be augmented by including the symbols 
used in the tail bits, which are also known to the receiver a priori. In the example 
below, a five tap filter is used (L = 5). Therefore, each row of the Z observation 
matrix comprises five imaginary and five real values of 

10 Letting y^e (A:) , w/?(/:) and bfi{k) denote the real part and yi{k), wi{k) and biik) 

denote the imaginary part of y{k), w{k) and b{k) respectively, then the optimal 

quasi-linear estimator w is solved by matrix inversion through the convolution of t' 
with Z in accordance with the above equation. In particular, the unknown impulse 
response w can be calculated through a well known relationship as the matrix product 



where R^z = Z^Z is the autocorrelation matrix, and p = Z^t' is the cross-correlation 
vector, and where 



15 



of 



w = Rip 



20 



w,(L-I) 



>v,(L-l) 



and 
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&,(62) 
bam 
b, (64) 
bA65) 



6,(86) 
fe«(87) 



and 





3',(62 + L 


-1) •• 


• ^,(62) 


y,{62 + L- 


1) •• 
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y,i63 + L 
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•• y.(63) 


-y,{63 + L- 


-1) - 
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-1) •• 
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5 Having determined w, an initial optimized signal estimate ^(^)can then be retrieved 



In this way, the filter is applied to the entire received data burst achieving co-channel 
10 and adjacent channel interference suppression through diversity in phase. After 
interference rejection, the equalized data may then be further processed in a known 
manner using Maximum Likelihood Sequence Estimation C^MLSE") techniques, or 
other known equalization methods, followed by soft-decision extraction, or soft 
decisions may be extracted directly. The process by which soft-decisions are 
15 generated is known, and is based on using an estimate of the residual interference and 
noise power to generate log-likelihood ratio (LLR) estimates for each symbol. 
To further enhance the interference reduction process, the observation matrix Z can 
be augmented by synthesizing additional data, which are not known a priori, using 
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the following steps before entering the equalization or direct soft-decision extraction 
process. 

Step 1: Process the received data using the quasi-linear estimator 

5 vector w to generate a vector of 1 16 log-likelihood ratio ("LLR") 

estimates corresponding to the burst data bits and stealing bits. 

Step 2: Inspect the resulting LLR estimates and identify a group of N 

bits based upon a predetermined condition. Examples of the 
10 predetermined condition include, but are not limited to: 

a) N bits having largest LLR magnitudes; 

b) N bits having largest LLR magnitudes which are larger than a 
threshold magnitude; and 

c) N bits having LLR magnitudes larger than a threshold 
15 magnitude. 

Step 3: Augment the observation matrix Z with the observation 

vectors, y(^), corresponding to the N bits identified in Step 2, and 
augment the training sequence vector, V with the resulting hard 
20 decisions determined by observing the LLR magnitudes of the N bits. 

Step 4: Re-compute the linear estimator vector, w, and re-process the 

received data using the re-computed linear estimator vector w to 
generate a vector of 116 log-likelihood ratio ("LLR") estimates 
25 corresponding to the burst data bits and stealing flags. 

Step 5: Return to Step 2, and repeat the steps if desired. 

Advantageously, the present invention does not require complex -conjugate 
30 manipulation of time or frequency shifting of signals to produce a corrected estimate 
of the desired signal. Moreover, linear combining of original, frequency-shifted, or 
complex-conjugate signals is not performed. Rather, the present invention makes use 



-9- 



CS23196RL 



EXPRESS MAIL NO.: EL977215603US 



of the phase redundancy of the GMSK waveform by alternatively extracting, 
according to reference timing established for processing the received burst, the real or 
imaginary part of the result of processing the received signal with the generally 
complex-valued taps of the equalizing filter. Therefore, non-linear means are used, 

5 founded through linear equalization methods. Further, the present invention is not 
based on considerations of cyclostationarity, but is instead constructed by making use 
of the signaMng phase occupancy redundancy present in GMSK waveforms. 

Optionally, the present invention can multiplex the real and imaginary 
components of the received signal upon entry to the filter. In summary, only the real 

iO or imaginary portion of the filtering operation is retained. It is therefore 

computationally wasteful to compute the portion that is not retained. This filtering 
operation is achieved through the multiplexing procedure described below. In brief, 
computations are performed for either (w/y^-C/:) + w,^y,(/:)) when the output is purely 
real or (w/ yy(A:) - w,^ yX^)) when the output is purely imaginary. This multiplexing 

15 can be used as a means for further reducing computational complexity and to increase 
the efficiency of the filter. This multiplexing is embodied in the present invention by 
noting that the output of the filter is alternately selected as the real and imaginary 
components. Noting that, if at a time k the filter output x{k) is expressed as: 



25 From the above, it can be seen that alternatively selecting the real and imaginary parts 
of the filter output can be viewed as equivalent to loading, in alternating fashion, the 
real and imaginary parts of the received signal y{k) into the real-valued filter 
apparatus used to form the respective inner products with the real and imaginary parts 
of the filter taps. In other words, if the data vectors loaded into the filters and w/ at 

30 time k are and y/, respectively, then the vectors loaded into Wr and W/ at time k+\ 
are y, and -yr. 



20 



= (yvr+jyvif{yr{k)-\-jyi{k)) 

= (w,-yw,/(y,W+;y,(A:)) 
= (w/ yr{k) + w/^ yik)) +y(w/ y^k) - w,^ y,i 



ik)) 
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Fig. 2 illustrates an exemplary receiver 200 with an alternate linear output 
equalizer ("ALOE") filter, for reducing interference in a desired signal in a GSM 
communication system in accordance with the present invention. In FIG. 2, thick 
lines represent the flow of complex values and thin lines represent real values. The 
5 front end of the receiver 200 has a conventional configuration, which is well known in 
the art, and therefore elements such as frequency converters etc. are not shown for 
simplicity. An analog-to-digital converter 202 ("ADC") is configured to convert an 
input signal that includes a desired signal and a co-channel interferer or adjacent 
channel interferer signal into an input digital waveform, y. Typically, one or more 

10 ADCs can be provided, such as for the digital conversion of in-phase (I) and 
quadrature (Q) data, as is known in the art, 

A burst synchronizer 204 is configured to receive the digital waveform, y, and 
to determine a synchronization delay, tq, between the signals from the desired user 
and the co-channel interferer. Generally, the synchronization delay, tq, includes an 

15 approximate or coarse synchronization from a tracking function, and a fine delay 
determination, both using techniques known in the art. A matrix kernel constructor 
206 is coupled to the burst synchronizer 204. The matrix kernel constructor 206 
assembles the observation matrix Z from the midamble portion of the input digital 
waveform and corrects for timing using the synchronization delay, tq. The matrix 

20 kernel constructor 206 then computes autocorrelation matrix = Z^Z and cross- 
correlation vector p = Z^t' . The present invention also envisions the use of one or 
more of leading zeros and ending zeros of a burst of waveform data for filter training 
since zeros also present a fixed phase relationship. The leading and ending zeroes can 
be used in addition to, or instead of, the midamble. 

25 The autocorrelation matrix, R^z , and the desired cross-correlation vector, p, 

are then passed to a filter parameter calculator 208 coupled to the matrix kernel 
constructor 206, The filter parameter calculator 208 is configured to receive the 
autocorrelation matrix, B^^z > and the desired cross-correlation vector, p, to define 
linear tap estimates, w, for the alternate linear output equalizer ("ALOE") filter 210. 

30 The ALOE filter 210, which is a finite-impulse-response ("FIR") filter, is coupled to 
and receives signals from both the ADC 202 and the filter parameter 208. The output 
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of the ALOE filter 210 is coupled to an output selector 212, which selects the signal 
path. The output selector 212 can select a first signal path for further signal 
processing in an equalizer such as a Maximum Likelihood Sequence Estimator 
C'MLSE"). For a second path, the output selector 212 is coupled to a log-likelihood 
5 ratio ("LLR") generator 214, which is configured to process the received signal y 
using the linear tap estimates w and the noise and interference power estinnate to 
generate a vector of 1 16 LLR estimates corresponding to the burst data bits and 
stealing bits. The noise and interference power estimate a' is generated by a noise 
power estimator 216 based upon information received from the burst synchronizer 

10 204. A bit selector 218 is coupled to the LLR generator 214, and is configured to 
receive the generated LLR estimates from the LLR generator 214. The bit selector 
218 inspects the generated LLR estimates, and identifies a group of N bits, a 
predetermined number of bits, based upon a predetermined condition. Examples of 
the predetermined condition include, but are not limited to: 

15 a) N bits having largest LLR magnitudes; 

b) N bits having largest LLR magnitudes which are larger than a 
threshold magnitude; and 

c) N bits having LLR magnitudes larger than a threshold magnitude. 
The selected N bits from the bit selector 218 are then passed back to the 

20 matrix kernel constructor 206 to augment the observation matrix Z with the 

observation vectors, y{k), corresponding to the N bits identified in the bit selector 218. 
With the training sequence vector, t', and optionally including the tail symbols, hard 
decisions corresponding to the LLR magnitudes of the N bits are provided. The linear 
tap estimate, v*^, is then re-computed in the filter parameter calculator 208, and the 

25 received data using the re-computed linear tap estimates, w, re-processed in the 
ALOE filter 210. If desired, the re-processed received data y(k) may be used to 
generate another vector of 1 16 log-likelihood ratio C*LLR") estimates corresponding 
to the burst data bits and stealing bits, and the process may be repeated. 

As detailed in FIG. 3, the FIR filter 210 is operable to input the tap 

30 parameters, w/, (shown here as five taps separated by one-symbol delays, z'') to 
operate on the waveform and also to output alternating real values of the real and 
imaginary components of the waveform to alternately linearly equalize the waveform 
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to provide an estimate of the desired signal. This equalized signal may then be used 
directly to determine soft decision values, or processed in an equalizer such as a 
Maximum Likelihood Sequence Estimator ("MLSE") for example, using techniques 
known in the art to output the estimated data sequence. 
5 In practice, the burst of waveform data is OMSK modulated, which is used to 

advantage for the training sequence in that the midamble consists of alternating real 
and imaginary symbols, which are extracted by alternate switching, for training the 
finite-impulse-response filter. The present invention can also make use of one or 
more of the three leading zeros and three tail zeros of a burst of waveform data as a 

10 portion of the filter training sequence since those zeros also present a fixed phase 

relationship. These can be used in addition to, or instead of, the midamble. The filter 
parameter calculator 208 minimizes the sum-squared error between a linear real 
estimate of the real and imaginary components of the training sequence of the filter 
and the received waveform, as explained previously. 

15 FIG. 4 is an exemplary flowchart 400 describing a method for reducing 

interference in a desired signal in a GSM communication system using an alternate 
linear output ("ALOE") filter, which is a finite-impulse-response filter, for alternate 
linear equalization, in accordance with the present invention. The method begins in 
block 402 by inputting a burst of data of a received waveform including interference 

20 from a channel of the communication system. In block 404, the ALOE filter is 

trained with a set of symbols of specific quadrature phase, known a priori, in the burst 
of data of the received waveform from block 402. For example, known real and 
imaginary components are alternatively selected from a midamble of a OMSK data 
burst. The set of symbols used for training, or the training symbols, can include one 

25 or more of leading zeros and ending zeros of the burst of data. The inclusion of the 
leading and/or ending zeroes can be in addition to the midamble or instead of the 
midamble. Preferably, the set of symbols used for training are the midamble of the 
burst of data. The use of the midamble of the burst is most applicable where the burst 
of data is GMSK-modulated since the midamble symbols only consist of a real or an 

30 imaginary-valued component, wherein the training step extracts the alternating non- 
zero real and imaginary components of the midamble symbols for training the ALOE 
filter. These components are then used in minimizing the sum-squared error between 
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a linear real estimate of the real and imaginary components of the training sequence of 
the ALOE filter and the received waveform. 

In block 406, the received waveform is operated on with the ALOE filter to 
alternately linearly equalize the burst of data to provide an estimate of the desired 
5 signal. Optionally, the operating step includes a substep of multiplexing the real and 
imaginary components of the received signal upon entry to the filter. In step 408, the 
ALOE filter output samples are processed to generate log-likelihood ratio ("LLR") 
values. These LLR values of the desired signal are then evaluated to determine 
whether they are satisfactory based upon a predetermined requirement in block 410. 

10 Examples of the predetermined requirement in block 410 include a preselected fixed 
number of iterations, comparison of current LLR values to previously obtained LLR 
values, or any other relevant conditions. If the resulting estimate is determined to be 
satisfactory, then the process terminates in block 410, which could, for example, 
represent further processing of the LLR values by a forward error correction decoder 

15 such as a convolutional code decoder. Otherwise, the resulting estimate of the desired 
signal is used to generate a vector of 1 16 log-likelihood ratio ("LLR") estimates 
corresponding to the burst bits and stealing bits in block 412. The generated LLR 
estimates are then evaluated based upon a predetermined condition. A group of N 
bits, which is a predetermined number, is also selected based upon the predetermined 

20 selection condition in block 414. Examples of the predetermined selection condition 
include, but not limited to, N bits having largest LLR magnitudes; N bits having 
largest LLR magnitudes which are larger than a threshold magnitude; and N bits 
having LLR magnitudes larger than a threshold magnitude. 

The selected N bits are then assigned values based upon hard decisions 

25 provided by the training symbols to augment the burst of data in block 416. The 

selected N bits are then passed back to block 404 augmenting the burst of data of the 
received waveform and the set of available training symbols, to re-train the ALOE 
filter, and the process is repeated. 



30 Example 

A numerical simulation was performed in accordance with the present 
invention. The results are provided below. Simulation data represents a single 
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random GMSK interferer in a typical urban (TU) environment, with a user velocity of 
3 kilometers per hour, where ideal frequency hopping is assumed. The logical channel 

uses the Advanced Multirate ("AMR") speech encoder at 12.2 kbps, as specified in 
known ETSI standards. A frame error rate was calculated for several carrier-to- 
interference ratios, 

FIG. 5 shows a graph 500 of the results indicating frame-error-rate ("PER") 
versus carrier/interference ratio ("C/I") in an operating environment as specified 
above. A conventional receiver (conventional MLSE Viterbi equalizer) is compared 
to a receiver incorporating the present invention using one-pass and two-pass 
processing. For the two-pass processing, additional six symbols identified as having 
the largest LLR magnitudes from the one-pass processing were used. As can be seen, 
the present invention provides an improved frame error rate 502 (shown as the left- 
most curve) over the prior art conventional receiver 504 (shown as the right-most 
curve), and over the one-pass processing receiver 506 (shown as the middle curve). 
In addition, this process is accomplished with much fewer calculations compared to 
the conventional receiver as described previously. 

In review, the present invention reduces co-channel, self and adjacent channel 
interference in a TDMA or GSM communication system, and in particular for GMSK 
modulation, using a finite-impulse-response filter alternate linear output equalization 
that reduces computational complexity without sacrificing performance. No 
additional hardware is required and processor resource requirements are reduced. 
While specific components and functions of the present invention are described 
above, fewer or additional functions could be employed by one skilled in the art and 
be within the broad scope of the present invention. The invention should be limited 
only by the appended claims. 
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